A correlation method and a block matching method based on a correlation calculation are known as conventional methods of detecting motion vectors between a plurality of images. The block matching method is a method of dividing an input image signal into a plurality of blocks of an appropriate size (8 pixels×8 lines, for example), calculating differences with pixels in a fixed range of a previous frame in units of these blocks, and searching for a block having a smallest sum of absolute values of the differences. A relative deviation between the images represents a motion vector of the corresponding block. The motion vectors of the respective blocks are then integrated, whereupon an inter-frame representative motion vector is calculated and used to perform final image registration.
A method of determining a sum of squared intensity difference (SSD), which is a sum of squares of a difference between pixel values, and a method of determining a sum of absolute intensity difference (SAD), which is a sum of absolute values of a difference between pixel values, are known as methods of searching for a block having a high correlation during block matching. In these methods, the correlation (degree of coincidence) is determined to be steadily higher as the SSD or SAD decreases. In a matching reference block region I and a subject block region I′, when pixel values of pixel positions p ε I and q ε I′ (where p, q are quantities having two-dimensional values, I, I′ are two-dimensional regions, and p ε I indicates that a coordinate p is included in the region I) are set as Lp, Lq, respectively, SSD and SAD are respectively expressed by following Equations (1) and (2).
                              SSD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                                                          ⁢                              q                ∈                                  I                  ′                                                              ⁢                                    (                              Lp                -                Lq                            )                        2                                              (        1        )                                          SAD          ⁡                      (                          I              ,                              I                ′                                      )                          =                              ∑                                          p                ∈                I                            ,                                                          ⁢                              q                ∈                                  I                  ′                                                              ⁢                                                Lp              -              Lq                                                                      (        2        )            
Further, a method employing a normalized cross-correlation (NCC) also exists. As a zero average correlation, average values Ave (Lp), Ave (Lq) of the pixels p ε I and q ε I′ respectively included in the matching reference block region I and the subject block region I′ are calculated, whereupon differences with pixel values included in the respective blocks are calculated using following Equations (3) and (4).
                              Lp          ′                =                                                                         Lp                -                                  Ave                  ⁡                                      (                    Lp                    )                                                                                                                    1                    n                                    ⁢                                                            ∑                                              p                        ∈                        I                                                              ⁢                                                                  (                                                  Lp                          -                                                      Ave                            ⁡                                                          (                              Lp                              )                                                                                                      )                                            2                                                                                                                            p            ∈            I                                              (        3        )                                          Lq          ′                =                                                                         Lq                -                                  Ave                  ⁡                                      (                    Lq                    )                                                                                                                    1                    n                                    ⁢                                                            ∑                                              q                        ∈                                                  I                          ′                                                                                      ⁢                                                                  (                                                  Lq                          -                                                      Ave                            ⁡                                                          (                              Lq                              )                                                                                                      )                                            2                                                                                                                            q            ∈                          I              ′                                                          (        4        )            
Next, the normalized cross-correlation NCC is calculated using a following Equation (5).NCC=ΣLp′Lq′  (5)
A block having a large normalized cross-correlation NCC is determined to be a block having a high degree of coincidence (a high correlation), whereupon a relative deviation between the blocks I′ and I having the highest degree of coincidence is determined and set as the motion vector.
When a motion vector is calculated in relation to an object that may be considered to be a stationary plain surface or may be in the infinite distance and substantially no effect is received from optical system distortion, the motion vectors of the individual regions match an overall motion vector of the image. However, when an image is obtained using an optical system that exhibits large distortion over a wide field angle, an amount of distortion varies greatly according to an image height, and therefore the motion vector of the image varies greatly according to the image height even when an object that may be considered to be a plain surface or may be in the infinite distance moves horizontally. For example, when an image is obtained using an optical system exhibiting negative distortion (barrel-shaped distortion), as in the case shown in FIG. 9, an image formed on an image periphery is smaller than an image formed in an image center, as shown in FIG. 10. It should be noted that the image height is a distance from an optical axis on an image surface.
As a result, it becomes difficult to perform image registration at a central portion and a peripheral portion of an image simultaneously when performing registration of a plurality of images, for example. To solve this problem, a technique of detecting a motion vector using an image in relation to which the optical system distortion has already been corrected is known (JP2007-129587A).